Scaling Techniques for $\epsilon$-Subgradient Methods

Nondifferentiable Optimization with Epsilon Subgradient Methods

Linear convergence of epsilon-subgradient descent methods for a class of convex functions

This paper establishes a linear convergence rate for a class of epsilon-subgradient descent methods for minimizing certain convex functions on R. Currently prominent methods belonging to this class include the resolvent (proximal point) method and the bundle method in proximal form (considered as a sequence of serious steps). Other methods, such as the recently proposed descent proximal level m...

Subgradient Methods

3 Convergence proof 4 3.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.2 Some basic inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.3 A bound on the suboptimality bound . . . . . . . . . . . . . . . . . . . . . . 7 3.4 A stopping criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.5 Numerical examp...

Subgradient methods for convex minimization

Many optimization problems arising in various applications require minimization of an objective cost function that is convex but not di erentiable. Such a minimization arises, for example, in model construction, system identi cation, neural networks, pattern classi cation, and various assignment, scheduling, and allocation problems. To solve convex but not di erentiable problems, we have to emp...

Stochastic Subgradient Methods

Stochastic subgradient methods play an important role in machine learning. We introduced the concepts of subgradient methods and stochastic subgradient methods in this project, discussed their convergence conditions as well as the strong and weak points against their competitors. We demonstrated the application of (stochastic) subgradient methods to machine learning with a running example of tr...